Reinforcement Learning Based Multi-Agent Resilient Control: From Deep Neural Networks to an Adaptive Law

Recent advances in Multi-agent Reinforcement Learning (MARL) have made it possible to implement various tasks cooperative as well competitive scenarios through trial and error, deep neural networks. These successes motivate us bring the mechanism of MARL into Resilient Consensus (MARC) problem that studies consensus a network agents with faulty ones. Relying on natural characteristics system goal, key component MARL, reward function, can thus be directly constructed via relative distance among agents. Firstly, we apply Deep Deterministic Policy Gradient (DDPG) each single agent train learn adjacent weights neighboring distributed manner, call Distributed-DDPG (D-DDPG), so minimize from suspicious eliminate corresponding influences. Secondly, get rid networks their time-consuming training process, Q-learning based algorithm, called Q-consensus, is further presented by building proper function credibility for pair update an adaptive way. The experimental results indicate both algorithms perform appearance constant and/or random agents, yet Q-consensus algorithm outperforms ones running D-DDPG. Compared traditional resilient strategies, e.g., Weighted-Mean-Subsequence-Reduced (W-MSR) or trustworthiness analysis, proposed has greatly relaxed topology requirements, reduced storage computation loads. Finally, smart-car hardware platform consisting six vehicles used verify effectiveness achieving velocity synchronization.